Image sensors in digital cameras, camcorders, and scanners are usually covered by a filter layer such that each pixel is sensitive to one color component only; for example, either red, green or blue. An example of such a filter arrangement is illustrated in FIG. 1, which shows a block of nine by nine pixels, each of which being associated with one of the red color channel (R), the green color channel (G), and the blue color channel (B). The arrangement of color filters shown in FIG. 1 is commonly known as a Bayer filter.
The raw data delivered by such an image sensor includes a plurality of pixels, wherein each pixel is associated with a single color channel, in accordance with the particular color filter mosaic employed. A substantial amount of processing is required for converting the raw image data to color image data, i.e., to assign a full RGB or YCbCr color value to each pixel. This processing may also be referred to as converting the image information from the CFA (Color Filter Array) domain to the multi-component-per-pixel domain, such as to the RGB (Red/Green/Blue) domain or to the YCbCr (luminance and blue/red color difference) domain. One of the major steps in this process is color interpolation, or demosaicing.
FIG. 2 is a schematic illustration of a conventional image processing pipeline, including steps for CFA pre-processing such as correcting defective sensor pixels and reducing Gaussian noise (S10), steps for applying digital gains (S20) such as antivignette and white-balance correction, steps for CFA post-processing (S40) and color interpolation (S50).
Under low-light conditions, the signal delivered by the sensor generally needs to be amplified by means of analog and digital gains. Analog gains are applied at the sensor level, whereas digital gains are applied in the digital domain by using appropriate multipliers that modify the signal in the digital domain.
The image-processing pipeline includes various blocks in which multipliers are applied to the input signal in order to perform some specific task. For example, one of the blocks of an imaging pipeline aimed at photographic quality is represented by the white balance block (S20) that is in charge of correctly balancing the color channels such that the final image looks correct in terms of colors and white-point representation, removing unwanted color casts. Another amplifier block using multiplications is the antivignette (S20) that corrects the light fall-off caused by the lens, especially in the peripheral areas of the image. The above two blocks usually work in the CFA domain. In terms of multiplications, the main difference between the two blocks is that white balance applies global digital gains whereas antivignette applies spatially varying digital gains. The color matrixing block is another pipeline element contributing with multiplications; however, this block is positioned after demosaicing and it is not taken into account in the following description for brevity.
Chroma noise usually appears in low-light conditions and manifests as colored blobs that are particularly evident in shadow areas where the signal-to-noise ratio can be extremely low. Existing solutions usually address the problem of chroma noise at the end of the pipeline, typically in the RGB domain or, better, in the YCbCr domain, allowing proper separation of luminance information from chrominance. In general, large kernels are required because of the low-frequency nature of the chroma blobs.
When reducing chroma noise in the YCbCr domain, the image has already undergone a substantial amount of processing. Gaussian noise reduction, antivignette, white balance, and color interpolation are among some of the algorithms that have already been applied before chroma-noise reduction takes place. Some of the aforementioned algorithms have low-pass effects, and others perform signal amplification either globally, locally, adaptively, or selectively. This yields an image in which the chroma noise has been amplified and undergone low-pass filtering. The result of this processing usually results in color blobs having low frequency that spread across many pixels of the image array. To effectively reduce the residual low-frequency noise, a conventional technique uses large kernels, possibly in combination with subsampling techniques. Domain changes, such as filtering in the wavelet domain are also viable. Each of these techniques, however, is rather complex both in terms of hardware and software implementation, and the results obtained still leave room for improvement.